Input processing method of mobile terminal and device for performing the same

ABSTRACT

An input processing method of a mobile terminal and a device performing the same are disclosed. Information on a user facility performed by a touch event generated at an area of the mobile terminal is compared with voice sensing data collected through a microphone to perform a specific user facility. As a result, erroneous touch event and imprecision generated during the voice sensing are improved.

CROSS RELATED APPLICATION(S)

This application is a continued application of U.S. Ser. No. 12/722,838,filed Mar. 12, 2010, which claims priority from an earlier Korean PatentApplication No. 10-2009-0035195 filed Apr. 22, 2009, the contents ofwhich are hereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to a mobile terminal and, moreparticularly, to an input processing method of compensating a touchevent and a voice sensing function in a mobile terminal to yield a moreprecise signal input, and an input processing device for performing thesame.

2. Description of the Related Art

Mobile terminals support voice call function and are widely used invarious fields for convenient use and easy mobility. Mobile terminalsprovide various inputting methods for ease of use. An existing mobileterminal provides a touch screen having a touch panel and a display unitsuch that the touch panel processes user manipulation for touching aspecific image output on the display unit by a user to generate acorresponding touch event and controls an application programcorresponding to a user facility based on the processing. However, thetouch screen function of the existing mobile terminal is used only whenthe display unit can be viewed or by only a user. Moreover, the touchevent processed by the touch panel is not precise so that a touch eventdemanded by a user may not be not correctly generated and applied.

SUMMARY OF THE INVENTION

The present invention provides an input processing method, performed ina mobile terminal, of removing imprecision of a touch function and avoice sensing function by operating respective elements of the mobileterminal and supporting a user to perform a desired correct input and toconveniently generate a specific input signal, and an input processingdevice using the same.

In accordance with an exemplary embodiment of the present invention, aninput processing device of a mobile terminal includes: a touch panel forgenerating a touch event; a microphone for collecting a voice signal; acontrol unit for sensing the voice signal and generating voice sensingdata, generating visual information text data to determine a userfacility corresponding to the touch event, comparing the voice sensingdata with the visual information text data to correct the voice sensingdata, and comparing the visual information text data with the voicesensing data to correct the visual information text data; and a displayunit outputting a screen corresponding to the user facility activatedbased on the corrected voice sensing data and the corrected visualinformation text data.

In accordance with an exemplary embodiment of the present invention, aninput processing method of a mobile terminal includes: collecting atouch event and a voice signal; sensing the voice signal to generatevoice sensing data; generating visual information text data to determinea user facility corresponding to the touch event; correcting the voicesensing data by comparing the voice sensing data with the visualinformation text data and correcting the visual information text data bycomparing the visual information text data with the voice sensing data;and performing the user facility based on the corrected voice sensingdata and the corrected visual information text data.

According to the input processing method and device of the presentinvention, operating precision of a user facility based on a touchfunction and a voice sensing function is improved and the user facilitycan be more conveniently and flexibly operated.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present invention will be moreapparent to those skilled in the art from the following detaileddescription in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic block diagram illustrating a configuration of amobile terminal according to an exemplary embodiment of the presentinvention;

FIG. 2 is a detailed block diagram illustrating a control unit of FIG.1;

FIG. 3 is a flowchart illustrating an inputting method of a mobileterminal according to an exemplary embodiment of the present invention;

FIG. 4 is an exemplary screen illustrating an inputting method of amobile terminal according to an exemplary embodiment of the presentinvention;

FIG. 5 is a flowchart illustrating an inputting method of a mobileterminal according to another exemplary embodiment of the presentinvention;

FIG. 6 is an exemplary screen illustrating an inputting method of amobile terminal according to a further exemplary embodiment of thepresent invention;

FIG. 7 is a flowchart illustrating an inputting method of a mobileterminal according to still another exemplary embodiment of the presentinvention;

FIG. 8 is an exemplary screen illustrating a user facility and anactivating method of the user facility in an inputting method of amobile terminal according to still another exemplary embodiment of thepresent invention;

FIG. 9 is an exemplary screen illustrating a phone call inputting methodof inputting method of a mobile terminal according to still anotherexemplary embodiment of the present invention;

FIG. 10 is an exemplary screen illustrating a message composing methodof an inputting method of a mobile terminal according to still anotherexemplary embodiment of the present invention; and

FIG. 11 is an exemplary screen illustrating a method of enabling a userfacility on a touch panel in an inputting method of a mobile terminalaccording to still another exemplary embodiment of the presentinvention.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments of the present invention aredescribed in detail with reference to the accompanying drawings. Thesame reference symbols are used throughout the drawings to refer to thesame or like parts. For the purposes of clarity and simplicity, detaileddescriptions of well-known functions and structures incorporated hereinmay be omitted to avoid obscuring the subject matter of the presentinvention.

Particular terms may be defined to describe the invention in the bestmanner. Accordingly, the meaning of specific terms or words used in thespecification and the claims should not be limited to the literal orcommonly employed sense, but should be construed in accordance with thespirit of the invention. The description of the various embodiments isto be construed as exemplary only and does not describe every possibleinstance of the invention. Therefore, it should be understood thatvarious changes may be made and equivalents may be substituted forelements of the invention.

Hereinafter, visual information text data are generated by a touch eventand may be characters or numbers corresponding to a text input, orcommand data for activating a user facility. Voice sensing data, inwhich voice is sensed based on collected voice signals, may be sensingdata for inputting text or command data for activating the userfacility.

FIG. 1 is a schematic block diagram illustrating a configuration of amobile terminal according to an exemplary embodiment of the presentinvention.

Referring to FIG. 1, a mobile terminal 100 according to an exemplaryembodiment of the present invention includes a radio frequency (RF) unit110, an input unit 120, an audio processor unit 130, a touch screen 140,a storage unit 150 and a control unit 160.

The mobile terminal 100 having the configuration may manage a functionof sensing voice collected through a microphone that is provided in theaudio processor unit 130 and converting the collected voice into text, afunction of outputting preset voice information stored in the storageunit 150 through a speaker, and a user facility performed by a touchevent collected through the touch screen 140, simultaneously orsequentially to enhance a precision of voice sensing data based on thetouch event and the ease of use of the touch event based on the voicesensing data so that a more precise input signal can be generated andcontrol of the user facility can be provided. Hereinafter, the elementsof the mobile terminal 100 will be described in detail.

The RF unit 110 generates communication channels for voice communicationand for transmission of data such as an image under the control of thecontrol unit 160. That is, the RF unit 110 generates a voicecommunication channel, a data communication channel, and a videocommunication channel between mobile communication systems. To this end,the RF unit 110 may include an RF transmitter for performingup-conversion and amplifying frequency of a transmitting signal and anRF receiver for performing low-noise amplification of a received signaland performing down-conversion of frequency of the received signal. TheRF unit 110 may be selected by the control unit 160 according to a touchevent generated by a touch panel 143 and may be activated based on thevoice sensing data generated by sensing voice during the selection. Thatis, the RF unit 110 may be activated or inactivated based on the touchevent and the voice sensing. For example, when a specific touch event isgenerated from the touch panel 143, and at the same time or after that,preset voice sensing data are collected within a preset time, the RFunit 110 may request a call based on a preset or input telephone numberor answer a received call.

The input unit 120 includes a plurality of input keys and function keysfor receiving number information or text information and setting variousfunctions. The function keys may include an arrow key, a side key, and ashort key for performing specific functions. The input unit 120generates a key signal in connection with user setting and functioncontrol of the mobile terminal 100 and transmits the generated keysignal to the control unit 160. The input unit 120 may be implemented bya QWERTY keypad, a 3*4 keypad, or a 4*3 keypad having a plurality ofkeys. The input unit 120 may be replaced with a touch screen 140 whenthe touch screen 140 of the mobile terminal 100 is implemented by a fulltouch screen.

The audio processor unit 130 includes a speaker SPK for reproducingaudio data transmitted and received for the call and a microphone MICfor collecting user voice and other audio signals during the call. Thevoice signal collected by the microphone MIC may be directly sensed as avoice by the control unit 160 according to whether a voice sensingfunction is set. The speaker SPK may output specific voice informationstored in the storage unit 150 as an audio signal under the control ofthe control unit 160. For example, when vocal information explaining aspecific user facility is provided and a corresponding user facility isassigned by a touch event, the speaker SPK may convert the vocalinformation into an audio signal to output.

The touch screen 140 includes a display unit 141 and a touch panel 143.The touch screen 140 is configured such that the touch panel 143 isdisposed on the front side of the display unit 141. The touch screen 140may have the same size as that of the touch panel 143.

In a preferred embodiment of the present invention, the display unit 141displays various menus of the mobile terminal 100 and informationincluding input by a user and to be provided to the user. That is, thedisplay unit 41 provides a plurality of screens generated when using themobile terminal 100, including a standby screen, a menu screen, amessage composing screen and a call screen. The display unit 141 dividesan entire displaying area into a plurality of divisional displayingareas each having a preset size and provides one of icons and imagescorresponding to the user facility on the plurality of divisionaldisplaying areas. For example, when the mobile terminal 100 provides awidget function and a message alarming function, the display unit 141divides the displaying area into a first and a second divisionaldisplaying area, and outputs at least one widget icon corresponding tothe widget function on the first divisional displaying area and one ofan image and an icon corresponding to the message alarming function onthe second divisional displaying area. The display unit 141 comprises aliquid crystal display (LCD), and an organic light emitting diode(OLED). The display unit 141 has a size less than that of the touchpanel 143 and is disposed under the touch panel 143.

In an exemplary embodiment of the present invention, the touch panel 143is disposed to cover the display unit 141, and generates a touch eventaccording to one of a contact by and a distance from an object totransmit the touch event to the control unit 160. In this case, thetouch panel 143 is a matrix detection type and transmits a positioninformation of a touch event generated on the touch panel 13 and a typeinformation of the touch event to the control unit 160. The control unit160 checks the position and type of the touch event transmitted from thetouch panel 143 and maps a specific image the position, and controls thedisplay unit 141 to activate a user facility linked to the image.Operation of the touch panel 143 will be described in detail later.

In a preferred embodiment of the present invention, the storage unit 150stores an application program required to operate each function employedin the exemplary embodiment of the present invention and one of a keymap and a menu map for operating the touch screen when the mobileterminal 100 includes a touch screen. There can be a plurality of typesof key maps and menu maps. That is, the key map may be a keyboard map, a3*4 key map, a qwerty key map, and an application control key map forcontrolling a currently activated application program. The menu map canbe one of a menu map for controlling the currently activated applicationprogram and a menu map having a plurality of items provided by themobile terminal 100. The storage unit 150 includes a program region anda data region.

The program region stores an operating system (OS) for booting themobile terminal 100 and operating the above-mentioned units and anapplication program reproducing a plurality of files, including anapplication program supporting a call function, a web browser accessingto Internet, an MP3 application program reproducing an MP3 file andother music files, an image outputting application program reproducing apicture and an image, and a moving picture reproducing applicationprogram according to the functions supported by the mobile terminal 100.The program region stores a touch operating program supporting a touchfunction, a voice sensing program supporting a voice sensing function,and a multiple input application program based on both the touchfunction and the voice sensing function. The multiple input applicationprogram substantially manages the touch function application program andthe voice sensing application program in a multiple way, and includes aroutine for simultaneously managing the touch event transmitted from thetouch function application program and the voice sensing datatransmitted from the voice sensing application program. The applicationprograms stored in the program region can be executed in a multipleinput mode of the touch function and the voice sensing function, and atouch function mode, and a voice sensing function mode.

The data region in which data generated when the mobile terminal 100 isused is stored and includes phonebook information, at least one icon forperforming the widget function, and a plurality of types of content. Thedata region, when the display unit 141 is a touch screen, can store auser input that is input through the touch screen. The data regionincludes a voice sensing database required to sense voice-sensed words.

The control unit 160 controls a power supply to the respective elementsof the mobile terminal 100 for performing initialization of the mobileterminal 100 and signal flow to multiply manage the touch function andthe voice sensing function when the initialization is completed. To thisend, the control unit 160, as illustrated in FIG. 2, includes a voiceprocessor unit 161, a touch processor unit 163, and a functionperforming unit 165, in an exemplary embodiment of the presentinvention.

The voice processor unit 161 controls the audio processor unit 130 tosense voice collected by the microphone MIC as voice based on the voicesensing database stored in the storage unit 150. The voice processorunit 161 transmits the voice sensing data to the function performingunit 165. The voice processor unit 161 controls the audio processor unit130 to output the voice data transmitted from the function performingunit 165 through the speaker SPK of the audio processor unit 130. Thevoice processor unit 161 manages at least one voice sensing algorithm.For example, in a voice sensing algorithm applied by the voice processorunit 161, a voice signal received by the microphone MIC is sampled at 16kHz and quantized with 16 bits to be stored. The quantized voice data isemphasized by a preset transfer function in advance and is multiplied byHamming window of 25 ms to be analyzed while moving every 10 ms. Bydoing so, a total 39 order voice characteristic parameter including adifference between a first order component and a second order componentcan be applied to a twelfth order LPC-MEL spectrum coefficient andnormalized algebra energy. A voice sensing model to which thecharacteristic parameter is applied employs a method of generating aphoneme decision tree at every state position of the model and learningstate sequence of a context dependent acoustic model using learningacoustic data by successive state splitting (SSS). Since this method isfast in performing state splitting, state is selected and split by theSSS simultaneously with splitting entire splittable state and statehaving maximum likelihood may be selected. The voice processor unit 161employs a hidden Markov network as an acoustic model. The voiceprocessor unit 161 applies a method of analyzing voice wave based on aplurality of algorithms to extract and splitting a voice band where avowel is characterized or characteristic equivalent thereto. The voiceprocessor unit 161 applies a plurality of voice sensing algorithms inaddition to the above-mentioned voice sensing algorithm.

The touch processor unit 163 transmits a touch event and positioninformation, transmitted from the touch panel 143 to the functionperforming unit 165. In this case, the touch processor unit 163determines which user function is assigned to the display unit 141 orselected based on a type of the touch event and the position informationand transmits the determined results to the function performing unit165. That is, the touch processor unit 163 determines whether the touchevent is one of a touch down event, a touch up event, a touch dragevent, and a flick event, and transmits information on the touch eventand position information where a corresponding touch event is generatedto the function performing unit 165. When the touch event is an event inwhich a touch moves a preset distance or longer on the touch panel 143such as a touch drag event and a flick event, the touch processor unit163 provides position information on a path of the movement.

In a preferred embodiment, the function performing unit 165 controls thevoice processor unit 161 and the touch processor unit 163 to processvoice sensing data and a touch event respectively according to a mode ofthe mobile terminal 100. In a mode where only voice sensing function isperformed, the function performing unit 165 processes the transmittedvoice sensing data independent from the touch function. Like this, in amode where only touch function is performing, the function performingunit 165 processes the transmitted touch event independently from thevoice sensing data. When the function performing unit 165 is set toprocess the touch function and the voice sensing function together, thefunction performing unit 165 checks whether the voice sensing data istransmitted within a preset time of when the touch event is transmittedand stores the transmitted touch event temporally. In other words, whena touch event, for example, one of event information on a text input, auser facility assignment, or a user facility selection is received fromthe touch processor unit 163, the function performing unit 165determines whether to perform the touch function based on a mode of themobile terminal 100 directly or in combination with the voice sensingdata transmitted from the voice processor unit 161. When a mode is setto process the touch function and the voice sensing function together,the function performing unit 165 processes the touch event and the voicesensing data respectively transmitted from the touch processor unit 163and the voice processor unit 161 together. In such a multiple mode, thefunction performing unit 165 checks whether a touch event is transmittedwithin a preset time after voice sensing data is transmitted andcontrols the user facility by processing the voice sensing data with thetouch event when the touch event is transmitted within the preset time.In this case, the function performing unit 165 uses one of a buffer,assigned by the storage unit 150, for storing the touch event and thevoice sensing data temporally and a memory provided in the control unit160.

When a touch event is received from the touch processor unit 163, thefunction performing unit 165 checks whether the touch event assigns aspecific user facility and may checks whether voice information is setto the specific user facility when the touch event assigns the specificuser facility. When there is voice information corresponding to the userfacility, the function performing unit 165 transmits the voiceinformation to the voice processor unit 161 and controls the voiceprocessor unit 161 to convert and output the voice information into anaudio signal using the audio processor unit 130.

Management of the mobile terminal 100 carried out by the functionperforming unit 165 will be described with a flowchart and exemplaryembodiments having exemplary screens in detail below.

FIG. 3 is a flowchart illustrating an inputting method of a mobileterminal according to an exemplary embodiment of the present invention.

Referring to FIG. 1, when electric power is supplied, the mobileterminal 100 initiates respective elements and outputs a preset standbyscreen on the display unit 141 when the initialization is completed(S101). In this case, the mobile terminal 100 outputs the standby screenon the display unit 141 even when the mobile terminal 100 is a sleepstate because the mobile terminal 100 has not been used for a presettime and is activated again. When the display unit 141 is activated, themobile terminal 100 activates the touch panel 143 according to a usersetting to prepare to sense a user touch.

Next, the mobile terminal 100 checks whether or not a voice sensingfunction is activated (S103). When the voice sensing function is notactivated, the mobile terminal 100 performs a user facility of themobile terminal 100 such as a call function, a message composingfunction, and a file reproducing function according to an input signalinput by a user, for example, a touch event transmitted from theactivated touch panel 143 (S105).

Meanwhile, when the voice sensing function is activated or an inputsignal for activating the voice sensing function is transmitted (S103),the mobile terminal 100 collects voice (S107). In this case, a user ofthe mobile terminal 100 generates a voice signal through the microphoneMIC of the mobile terminal 100. Then, the mobile terminal 100 performsthe voice sensing function with respect to the voice signal using theabove-mentioned voice sensing algorithms. When the voice sensing iscompleted, the mobile terminal 100 generates voice sensing data based onthe voice sensing. The voice sensing data can be text information. Whenthe sensed voice is not correct, the mobile terminal 100 sets aplurality of voice sensing data as candidates for the voice sensingresults. For example, when a user pronounces “Thank you”, the mobileterminal 100 can set “Thank you”, “Thunk kou”, “Thank kou”, etc. ascandidates.

Next, the mobile terminal 100 collects touch events generated from thetouch panel 143 (S111). To this end, a user of the mobile terminal 100touches the touch panel 143 using any one of his/her finger, a styluspen, and other tool. The mobile terminal 100 performs the voice sensingand determines a touch event generated at the touch panel 143 within apreset time as an available touch event required for multiple operation.That is, when the touch event is generated after the preset time hasbeen lapsed, the mobile terminal 100 makes the generated voice sensingand the touch event generated thereafter void or to be processedindependently.

When touch events are collected from the touch panel 143, the mobileterminal 100 compares the voice sensing with results of the touch eventsto produce an appropriate result (S113). That is, the mobile terminal100 selects a result in which text information corresponding to thevoice sensing data is matched with information sensed from the touchevent. Candidates of the voice sensing data among the above-mentionedexample are provided, and the mobile terminal 100 determines whetherthere is a most similar information with the information sensed from thetouch events among the candidates of the voice sensing data. The mobileterminal 100 controls a current user facility of the mobile terminal 100with reference to the result obtained by comparing the voice sensingwith the touch event (S115).

The multiple operations of the touch function and the voice sensingfunction in the above-mentioned exemplary embodiment of the presentinvention will be described with reference to FIG. 4, in detail.

FIG. 4 shows an exemplary screen illustrating a multiple operation of atouch function and a voice sensing function according to an exemplaryembodiment of the present invention.

Prior to description, hereinafter a user facility of a mobile terminalwill be described by considering a text inputting function as anexample.

Referring to FIG. 4, a user of the mobile terminal 100 sets a mode forthe multiple operation of the voice sensing function and the touchfunction after the mobile terminal is powered on. Then, the mobileterminal 100, as shown in a screen 101, activates the voice sensingfunction according to a mode setting. In this case, the mobile terminal100 outputs an image indicating that the voice sensing function iscurrently activated on the display unit 141. The image displayed on thedisplay unit 141 may be omitted by one of a user setting or a designer'sintent. A user of the mobile terminal 10 may provide a preset voicesignal, for example, “Thank you for your message Su-jan” to an areawhere the microphone MIC of the mobile terminal 100 is positioned. Then,the mobile terminal 100 collects the voice signal using the microphoneand performs the voice sensing with respect to the collected voicesignal using the stored voice sensing algorithm. During the processing,the mobile terminal 100 generates candidates for the voice sensing data.In other words, the mobile terminal 100 may not correctly sense aspecific voice signal. In this case, the mobile terminal selects aphoneme or a syllable similar to the voice signal from the voice sensingdatabase in order to improve voice sensibility, wherein a plurality ofsimilar phonemes or syllables may be selected. The mobile terminal 100may construct an existing word based on the sensed phonemes or syllablesby referring the voice sensing database. In this case, when a pluralityof phonemes or syllables is selected with respect to a specific voicesignal, a plurality of words may be selected. The mobile terminal 100may select the plurality of words or a sentence based on the words ascandidates of the voice sensing data. The mobile terminal 100 may notselect candidates of the voice sensing data but may select one specificvoice sensing data based on a set algorithm even when an error isgenerated.

After that, the user of the mobile terminal 100, as shown in a screen103, generates a touch event for inputting a visual information textinput using an object selected from the group consisting of a styluspen, a finger, and other tool to touch the touch panel 143. To this end,the user may activate a text input window of the mobile terminal 100.The user may input visual information, that is, a text by keeping apreset direction and distance while continuously generating a touchmovement or a touch drag signal after touching the touch panel 143 witha finger. The visual information includes a plurality of types ofpatterns, pictures, symbols, images, letters, numbers, and icons, andthe user may input various information included in the visualinformation through the touch panel 143. Hereinafter, for theconvenience of description, it is assumed that the user inputs textcorresponding to a specific letter or number on the touch panel 143. Themobile terminal 100 collects touch event information input by the userand determined whether a corresponding touch event is matched with aspecific text. The mobile terminal 100 determines what letter the touchevent generated at a text input window form. For the determination, themobile terminal 100 prepares a text database used as a reference todetermine the touch events collected from the touch panel 143 as a text.That is, when the touch events collected from the touch panel 143 formsa certain letter, the mobile terminal 100 determines which of a letterand number a corresponding letter is matched with by referring to thetext database. When the touch event-based text sensing is completed andthe text data is generated, the mobile terminal 100 compares the voicesensing data as illustrated in the screen 101 with the text data asillustrated in the screen 103. Information stored in the text databaseis not limited to text corresponding to specific letters or numbers.That is, the text database store a plurality of visual informationincluding various letters, numbers, symbols, pictures, patterns, andicons, and the mobile terminal 100 determine which visual information ortext of the text database a touch event sensed by the touch panel 143corresponds to.

During the comparison, the mobile terminal 100 selects the most similardata to the text data sensed by the touch event from the candidates ofthe voice sensing data and outputs the selected data on the display unit141 as illustrated in a screen 105. The selection of the most similardata is made by comparing letters or numbers existing in the candidatesof the voice sensing data with the text data to select a word or asentence including the largest number of identical letters or numbers.Similar to the voice sensing, in sensing a letter based on the touchevent, the letter sensed by the touch event may be a plurality of types.For example, although a user generates a touch event for inputting aletter corresponding to ‘a’, the mobile terminal 100 may sense the touchevent as ‘u’ or ‘o’. Thus, the touch event may select a plurality ofwords as text data candidates during the letter sensing from the textdatabase. After that, the mobile terminal 100 may compare the voicesensing data candidates with the text data candidates and may select ahighly similar word or a sentence from the compared results. When onlyone voice sensing data is selected, the mobile terminal 100 selects themost similar text data from the text data candidates based on theselected voice sensing data. Consequently, the mobile terminal increaseprecision of the voice sensing data based on the text data sensed fromthe touch panel 143 or precision of the text data sensed from the touchpanel 143 based on the voice sensing data. In more detail, it isdifficult, in case of voice sensing, to sense precise voice due tovarious circumstance effects such as ambient noise and other voice wavessensing, and it is difficult, in case of text input, to sense precisetext because users of mobile terminals have various writing habits, forexample, handwritings. In this situation, the mobile terminal 100 mayimprove precision of the voice sensing function and the touch functionby comparing the voice sensing data with the text data sensed by thetouch event and selecting a phoneme, a syllable, a word, or a sentencehaving the most similarity.

FIG. 5 is a flowchart illustrating an operation of a mobile terminalusing an inputting method of a mobile terminal according to anotherexemplary embodiment of the present invention.

Referring to FIG. 5, when electric power is supplied, a mobile terminal100 initiates respective elements of the mobile terminal 100 and outputsa preset standby screen on the display unit 141 after the initialization(S201) is completed. In this case, the mobile terminal 100 outputs thestandby screen on the display unit 141 even when the mobile terminal 100is a sleep state because the mobile terminal 100 has not been used for apreset time and is activated again.

Next, the mobile terminal 100 checks whether the touch panel 143 isactivated (S203). In a case where the mobile terminal 100 is providedwith a touch screen, since the touch panel is automatically activatedafter the initialization or when a standby screen is output, step S203may be omitted. When the automatic activation of the touch panel is notset or the touch panel is needs to be activated, a user of the mobileterminal 100 transmits an input signal to activate the touch panel usingan input unit. When there is no signal for activating the touch panel,the mobile terminal 100 performs a user facility, for example, variousfunctions provided by the mobile terminal 100 selected from the groupincluding a file reproducing function, a camera function, and abroadcast receiving function according to an input signal transmittedfrom the input unit (S205).

When the touch panel is activated (S203), the mobile terminal 100collects touch events generated by touch on the touch panel (S207). Tothis end, the user may generate on the touch panel with a finger.

When the touch events are collected, the mobile terminal 100 processes acorresponding touch event and outputs the processed result (s209). Inmore detail, the mobile terminal 100 checks what letter or number thecorresponding touch event indicates when the touch event is to input aspecific text. For the checking, the mobile terminal 100, as describedabove, may refer to the text database. When the touch event is generatedon a specific image or at a specific area of the display unit to whichan input signal to activate a user facility corresponds, the mobileterminal transmits information on which the corresponding touch event isassigned to or selects a specific image or an area of the display unitto the function performing unit. To this end, the mobile terminal 100includes text data defining a specific image of the display unit 141. Bydoing so, the mobile terminal 100 prepares text data candidates for animage output on the display unit 141 with respect to a result of thetouch event. For example, in the assignment of a specific image or anarea of the display unit 141, the mobile terminal 100 sets a pluralityof corresponding areas as the text data candidates when a plurality ofareas is selected due to shaking or other incorrect motion or a touchevent approaching a specific image area is generated. Alternately,although a specific image area is not correctly touched, other imageareas near the area where the touch event is generated may be set as thetext data candidate. The mobile terminal 100 generates the text datacandidates for a touch event corresponding to the writing of letters ornumbers.

Next, the mobile terminal 100 checks whether voice is input (S211). Whenvoice is not input, the mobile terminal 100 output a processing resultof the touch event (S209). The mobile terminal 100 automaticallyactivates the voice sensing function before, during, or after performingstep S209. That is, the mobile terminal 100 activates the voice sensingfunction before performing step S211.

When voice is input (S211), the mobile terminal 100 performs the voicesensing with respect to the input voice (S213). In this case, the mobileterminal 100 performs the voice sensing to the input voice in a phoneme,a syllable, a word, a phrase, and a sentence and transmits voice-senseddata to the function performing unit 165. The mobile terminal 100 makesvarious voice sensing data candidates for incomplete voice sensing.

Next, the mobile terminal 100 compares the text data candidates with thevoice sensing data candidates to select data having the most similarityand controls a user facility according to a corresponding result. Forexample, when a touch event result is a text input for inputting aspecific letter or number, the mobile terminal 100 corrects a part ofthe input text where a typing error is generated based on the voicesensing data. The mobile terminal 100 selects an image very similar tothe voice sensing data from the text data candidates defining an imageon the display unit 141. For example, when a user selects a first areaand a second area of an image on the display unit 141 duplicately, themobile terminal 100 stores text information on the first and second areaas the text data candidates, determines that the user desires to selectthe first area when voice sensing data corresponding to the first areais sensed, and performs a user facility corresponding to the first area.

In the above description, although the voice sensing data candidates andthe text data candidates are set, compared with each other, and datahaving the most similarity is selected, the mobile terminal 100 mayhandle the text data candidates without setting the voice sensing datacandidates or vice versa. This will be described with reference to FIG.6 in detail.

FIG. 6 shows an exemplary screen illustrating an inputting method of amobile terminal according to another exemplary embodiment of the presentinvention. Hereinafter, a text inputting function of a mobile terminalwill be described. The text inputting function corresponds to a part ofuser facilities of a mobile terminal selected from the group consistingof a function of composing a message or an e-mail message, a schedulingfunction, and a file-naming function.

Referring to FIG. 6, a user of a mobile terminal 100 activates a textinputting function for inputting a specific text. For the textinputting, the user selects the text inputting function from the menufunction of the mobile terminal 100 or clicks a hot key when the hot keyis assigned. When the text inputting function is activated, the mobileterminal 100, as shown in a screen 201, provides a screen such that theuser inputs text on the entire surface of the touch screen by generatingtouch events. Then, the user draws a path, corresponding to a specificletter or number, on the touch screen using one of a touch-down event, atouch-move event, and a touch-up event. Then, the mobile terminal 100outputs one of a letter and number corresponding to the touch event onthe touch screen. In the drawing, the mobile terminal 100 outputscursive letters corresponding to a letter image “Thank you”corresponding to the touch event of the user's.

When a user inputs a letter and one of a preset time has elapsed and aninput signal corresponding to the letter input is generated, the mobileterminal 100 senses the letter drawn on the touch screen. Then, themobile terminal 100, as shown in a screen 203, outputs a touch sensingresult of an image input on a screen 201. However, the mobile terminal100 may erroneously sense the cursive letter image on the screen 201 as“Thunk kou.”

After the touch event is completely sensed, the mobile terminal 100 mayactivate the voice sensing function for collecting voice within a presettime and may collect voice signals corresponding to “Thank you” providedby the user as shown in the screen 203.

The mobile terminal 100 performs the voice sensing to “Thank you”,corrects Thunk kou” sensed from the touch event according to the voicesensing data into “Thank you”, and outputs the correct result to thetouch screen as show in a screen 205. During the voice sensing, themobile terminal 100 sets various candidates for “Thank you” and selects“Thank you” having the most similarity with “Thunk kou” sensed from thetouch event.

As described above, in the inputting method according to an exemplaryembodiment of the present invention, a touch event sensing result isobtained even when a sensed part has an error without setting text datacandidates. The mobile terminal 100 reduces incorrectness generatedduring the voice sensing and the touch event sensing by selecting onevoice sensing data from various voice sensing data candidates set duringthe voice sensing based on a result obtained by the touch event.

FIG. 7 is a flowchart illustrating an inputting method of a mobileterminal according to still another exemplary embodiment of the presentinvention.

Referring to FIG. 7, in an inputting method of a mobile terminalaccording to still another exemplary embodiment of the presentinvention, firstly the mobile terminal 100 activates a touch panel 143while preparing the mobile terminal 100 to enable a user to use themobile terminal 100 by one of initializing the mobile terminal 100 orreleasing a sleep state (S301). In this case, the mobile terminal 100activates only the touch panel 143 included in a touch screen 140. Thatis, the mobile terminal 100 may activate the touch panel 143 only whilekeeping the display unit 141 turned off or activate the touch panel 143and the display unit 141 simultaneously.

Next, the mobile terminal 100 may collect touch events generated at theactivated touch panel 143 (S303). To this end, a user of the mobileterminal 100 touches the touch panel 143 with one of a finger and othertool.

When the touch events are collected, the mobile terminal 100 outputs aspecific function in the form of sound based on the collected touchevents. To this end, a user generates a touch event in a preset specificarea on the display unit 141 such that a function on a user facility isoutput in sound. For example, when the display unit 141 is activated,the user may assign a touch event to an image corresponding to aspecific user facility and the mobile terminal 100 may output voiceinformation set to the assigned image. For example, when the image isassigned to a “calling function of RF unit functions” and the usergenerates a touch event on the image, the mobile terminal 100 outputsvoice information such as “Do you perform a calling function?”

Next, the mobile terminal 100 checks whether a touch event for selectingor activating a specific function or voice is collected (S307). When thetouch event or the voice collection is not generated, the mobileterminal 100 continuously performs a function of outputting descriptionfor a user facility according to a touch event in sound (S305).

When a touch event or voice for activating and selecting a specific userfacility is collected (S307), the mobile terminal 100 performs theselected user facility (S309). In this case, the mobile terminal 100performs the collection of the touch event and voice multiply orindependently. In more detail, although voice information is outputaccording to the user facility assignment or the voice information iscompletely output, the mobile terminal 100 activates a correspondinguser facility when a touch event for activating the corresponding userfacility on a specific image where the assignment is maintained, forexample, a ‘double tap’ is generated. When a corresponding voice sensingis completed after voice corresponding to “execution” is collected froma user in a state when the voice information is being output orcompletely output according to the user facility assignment, the mobileterminal activates the assigned user facility based on the voice sensingdata. The mobile terminal 100 may activate the corresponding userfacility after both a touch event as an input signal for activating theuser facility and the voice sensing is input. In this case, when thetouch event and the voice sensing are performed one of simultaneously orsequentially within a preset time, the mobile terminal 100 activates thecorresponding user facility.

In the above description, the term “assignment” may refer to a statewhen a user views an image output on the display unit 141, that is, astate before the user facility assigned to the image by highlightingedges of the image is activated. The term “selection” may refer to astate when the user facility assigned to the image is activated.

The inputting method according to still another exemplary embodiment ofthe present invention will be described in detail with reference toFIGS. 8 to 11.

FIG. 8 shows an exemplary screen illustrating a user facility alarmingand an activating method of the user facility in an inputting method ofa mobile terminal according to still another exemplary embodiment of thepresent invention. Hereinafter, the user facility will be described byan example of a phone book function.

Referring to FIG. 8, the display unit 141, as shown in a screen 301,includes a main indicator area 81, a phone book area 83, a contact area85, a list area 82, an eleventh menu area 87 a, a twelfth area menu area87 b, a thirteenth menu area 87 c, and a fourteenth menu area 87 d. Whenthe screen 301 is output, the mobile terminal 100 activates the touchpanel 143 of the touch screen 140. The mobile terminal 100 sets one ofvoice signals to be output to the respective areas and text informationto be convertible into voice signal. For example, when a user generatesa touch event by touching the main indicator area 81 on the touch panel143, the mobile terminal 100 outputs a preset voice signal indicatingthe touched area is the main indicator area 81 using the audio processorunit 130. Similarly, when a user generates a touch event by touching anarea of the touch panel 143 where a specific item, for example, “Mary”is output in the list area 82, the mobile terminal 100 outputs a voicesignal corresponding to the term “Mary” using the audio processor unit130. At this time, the mobile terminal 100 uses the Text to Signal (TTS)function as a function of converting and outputting text informationinto a voice signal. The TTS function converts an input text letter intoa preset voice signal and combining the voice signals to mechanicallypronounce a specific word and sentence. Thus, the mobile terminal 100assigns text information to the respective areas of the screen 301 andconverts the text information into a voice signal when a correspondingarea is selected.

In addition to a case when a user touches the respective areas of thedisplay unit 141 individually, when the user generates touch events on aplurality of the areas sequentially, the mobile terminal 100 assignsvoice signals corresponding to the sequential touch events. For example,when a user touches the phone book area 83 and moves a touching toolsuch as a finger or a stylus pen in a touch down state to perform atouch drag travelling list items, for example, from the item “Kevin” tothe fourteenth menu area 87 d via the items “Snoopy”, the eleventh menuarea 87 a, the twelfth menu area 87 b, and the thirteenth menu area 87c, the mobile terminal 100 outputs a preset voice signal assigned to thephone book area 83 such as “This is a phone book area.” Moreover, themobile terminal 100 outputs voice signals “This is Kevin.” and “This isSnoopy.” The mobile terminal 100 outputs a voice signal such as a voice“This is Create contact” assigned to a function assigned to the eleventhmenu area 87 a such as “Create contact” function. Similarly, when thetwelfth menu area 87 b is assigned to a “Delete” function and thethirteenth menu area 87 c is assigned to a “More” function, the mobileterminal 100 outputs voice signals assigned to the respective menu areas87 b and 87 c such as “This is a Delete function” and “This is a Morefunction.” Finally, when a touch event is generated in the fourteenthmenu area 87 d, the mobile terminal 100 outputs a voice signal “This isa Group view function.” assigned to a function such as the “Group view”function corresponding to the fourteenth menu area 87 d. The voicesignal is output when a touch event is generated at a correspondingarea. Thus, when the touch drag event travels through a specific areafaster than a voice signal output for a preset time, outputting time ofthe voice signals corresponding to the respective areas may beoverlapped. In this case, the mobile terminal 100 may preventoverlapping of the voice signals. That is, when a touch event isgenerated in another area while a touch event is generated and acorresponding voice signal is output, the mobile terminal 100 interruptsthe output of a voice signal assigned to a previous area and outputs avoice signal assigned to an area where another touch event is currentlybeing generated.

A user of the mobile terminal 100 may perform a preset voice input afterhearing which user facilities are assigned to the respective areas inthe screen 301 displayed on the display unit 141 through voice signalsby generating touch events. That is, when a touch event is generated inthe display unit 141 and the touch event is maintained in a specificarea, the user may input a specific voice signal corresponding to afunction assigned to the specific area, for example, the item “Groupview” through the microphone MIC. Then, the mobile terminal 100 maycompare text information of the area where the touch event is maintainedand voice sensing data of the input voice signal to determine whetherthe text information is matched with the voice sensing data, and mayshift from the screen 301 to a screen corresponding to the item “Groupview” as shown in a screen 303 when the text information is matched withthe voice sensing data. The screen 303 supports the item “Group view”and include areas selected from the group consisting of the mainindicator area 81, the phone book area 83, a group area 84, a group listarea 86, a twenty first menu area 86 a, and a twenty second menu area 86b. The group area 84 indicates a screen generated based on the voicesensing data and the group list area 86 is provided to output groupinformation classified and stored in a phone book. The twenty first menuarea 86 a and the twenty second menu area 86 b support specific menufunctions within the group view menu wherein the twenty first menu area86 a supports a “Create group” function and the twenty second menu area86 b supports the “Delete” function. In the Group view screen, as shownin the screen 301, voice signals or text information convertible intovoice signals may be assigned to the respective areas. Hereinbefore,when a voice sensing is generated in maintaining a touch event on aspecific area, text information assigned to an area corresponding to thetouch event is compared with voice sensing data, but the presentinvention is not limited thereto. In other words, when text informationassigned to the fourteenth menu area 87 d or a voice signalcorresponding to a specific function is input after a touch up event isgenerated in an area when the touch event is completed, for example, thefourteenth menu area 87 d, the mobile terminal 100 performs a menucorresponding to the fourteenth menu area 87 d. Moreover, the mobileterminal 100 compares the voice sensing data with overall text data orvoice signals assigned to areas through which the touch drag passes andmay activate a user facility assigned to a corresponding area when thereis text data or a voice signal corresponding to the voice sensing data.

As described above, when a touch event is generated already or currentlyin at least one area corresponding to a specific user facility in thedisplay unit 141 and voice sensing data obtained by collecting voice istransmitted, the mobile terminal 100 determines whether at least textinformation or voice signal assigned to the at least one area is matchedwith the voice sensing data. When matched up, the mobile terminal 100activates the user facility assigned to the area. By doing so, a touchevent caused by an incomplete touching action and incorrect voicesensing are mutually compensated with each other so that a user facilitycan be executed exactly.

FIG. 9 is an exemplary screen illustrating a phone call inputting methodof the inputting method of a mobile terminal according to still anotherexemplary embodiment of the present invention. The phone call inputtingmethod will be described with reference to a screen where the phone bookfunction is activated. However, the present invention is not limitedthereto but may be applied to a call log screen.

Referring to FIG. 9, when a user selects a specific receiver, forexample “Adam Smith” from the list area 92 to which various itemsincluded in the phone book are output as shown in a screen 311, themobile terminal 100 outputs a user facility executed based on theselected item as a voice signal. When a specific touch event sweepingright to left is generated at the item “Adam Smith,” the mobile terminal100 outputs a preset voice signal “This is Adam Smith.”

After that, when the mobile terminal 100 senses a specific voice signalinput by a user such as “View information” and outputs a screen 313corresponding to the “View information.” Referring to the screen 313,the mobile terminal 100 outputs a screen including any of a title area91 corresponding to the item “Adam Smith”, a photo area 93 when thephone book includes photos, a first phone number area 95 correspondingto the selected item, and a second phone number area 97.

In the screen 313, when the user wishes to call “Adam Smith,” the usergenerates a specific touch event on the first phone number area 95 orthe second phone number area 97, for example, a touch event sweepingright to left within a corresponding area. Then, the mobile terminal 100performs a procedure of establishing a call based on a phone numberassigned to the first phone number area 95 in response to the specifictouch event.

The mobile terminal 100 searches for a specific item included in thephone book during the outputting of phone book information. To this end,the mobile terminal 100 provides a search window to a preset region ofthe display unit 141. Meanwhile, a user of the mobile terminal 100generates a touch event sweeping a preset touch event such as the searchwindow in the region at which the search window is generated. Then, themobile terminal 100 outputs a voice signal “Item is searched for.” Inthis case, when the user provides a voice signal corresponding to a nameof the specific item such as “Adam Smith,” the mobile terminal 100senses the voice signal on the item “Adam smith” and reduces a voicesensing range and improve precision of the voice sensing by restrictingthe voice sensing range to the items stored in the phone book. Themobile terminal 100 senses the voice signal on a specific item anddisplays detailed information included in an item corresponding to thevoice sensing data when the item is detected.

Although a specific touch event has been described by the right-to-leftsweeping within a specific region, the present invention is not limitedthereto. That is, a touch event may include any of a flick touch event,a plurality of tap touch events, and a long press touch event. When atleast one touch event is generated in a corresponding area, the mobileterminal 100 activates a user facility assigned to the correspondingarea.

As described above, when a specific voice signal is input after a touchevent is generated at a specific area, the mobile terminal 100 executesa command corresponding to the voice signal. That is, when a touch eventon a specific item is generated in one of the phone book and the calllog and a voice signal is input, the mobile terminal 100 executes aspecific command based on voice sensing data of the input voice signal.In this case, the mobile terminal outputs a voice signal set to an areawhere the touch event is one of generated and reproduced based on thetext information corresponding to the voice sensing data.

FIG. 10 is an exemplary screen illustrating a message composing methodof the inputting method of a mobile terminal according to still anotherexemplary embodiment of the present invention. Hereinafter, the messageinputting method will be described by an example of a screenillustrating a phone book function. However, the present invention isnot limited thereto but may be applied to various call-related screensincluding the call log screen, a message receiving screen, and a messagesending screen.

Referring to FIG. 10, the mobile terminal 100 outputs a list area 102 ofthe phone book function as shown in a screen 321. A user of the mobileterminal 100 selects user information corresponding to a messagereceiver through a screen to which the list area 102 is output. Forexample, the user selects the item “Catherine” and performs a touchevent including a message icon 103 to which a user facility shift isassigned when a specific user facility is used based on the selecteditem. That is, the user performs a touch event sweeping right to left inthe area to which the item “Catherine” is output and moves the sweeptouch event through the message icon 103. Then, the mobile terminal 100senses the selections of the item “Catherine” and the message icon 103.Inn this case, the mobile terminal 100 outputs a voice signal assignedto the selection of the item “Catherine,” for example, “This isCatherine.” Moreover, the mobile terminal 100 outputs a voice signalcorresponding to “Do you compose a message to Catherine?” according tothe selections of the item “Catherine” and the message icon 103.

When the user gives an affirmative answer, for example generates a voicesignal corresponding to “Yes”, the mobile terminal 100 collects andsenses the voice signal and outputs a message composing window as ascreen 323.

In the above description, the user performs the touch event on the item“Catherine” and generates the touch event including the message iconassigned to a right side of the item, but the present invention is notlimited thereto. In other words, when a touch event touching only theitem “Catherine” regardless of inclusion of the message icon 103 intothe touch event, the mobile terminal 100 outputs the voice signal, thatis, the voice signal corresponding to “Do you compose a message toCatherine?” Meanwhile, when a touch event including the message icon 103and a call icon is generated, the mobile terminal 100 outputs a voicesignal such as “Message composing to and call to Catherine are enabled.”When a voice signal corresponding to one of “Message composing” and“Phone calling” is received from a user, the mobile terminal 100activates a user facility according to the voice signal. The mobileterminal 100 searches for voice sensing data corresponding to “Messagecomposing” or “Phone calling” during the voice sensing of the voicesignal so that a voice sensing range is reduced and it is clearlydetermined which user facility the voice sensing data assigns even whenan error occurs.

As described above, after a touch event is generated at a specific itemincluding a phone number or when a specific voice signal is sensed whilemaintaining the touch event, the mobile terminal 100 executes a specificuser facility based on the executed voice sensing data. Thus, the mobileterminal 100 restricts a voice sensing range to a range of userfacilities executed based on an item assigned by a touch event so thatpossibility of imprecision occurring during the voice sensing can bereduced.

FIG. 11 is an exemplary screen illustrating a method of enabling a userfacility on a touch panel in the inputting method of a mobile terminalaccording to still another exemplary embodiment of the presentinvention.

Referring to FIG. 11, a screen 331 indicates a state when electric poweris not supplied to a display unit 141 but supplied to a touch panel 143disposed in an upper side of the display unit 141 of the mobile terminal100. A screen of the display unit 141, as shown in a screen 333, isdivided into several regions of a preset size to which user facilitiesare assigned. In a state where the screen 331 is displayed, when thedisplay unit 141 is powered, the mobile terminal 100 output icons orimages corresponding to the user facilities as shown in the screen 333.

The mobile terminal 100 divides the touch panel 143 activated in thestate as shown in the screen 331 into the regions as shown in the screen333. That is, the mobile terminal 100 divides the touch panel 143 asshown in the screen 331 into a first region 111, a second region 113,and a third region 115. The display unit 141 as shown in the screen 333is divided into a first area 111, a second area 113, and a third area,wherein the first area 111 is an indicator area, the second area is awidget area, and the third area 115 is an unchecked event alarming area,for example.

In the screen 331, when a user generates a preset touch event at thefirst area 11, for example, a touch event sweeping right to left orother various touch events, the mobile terminal 100 outputs a presetvoice signal assigned to a corresponding to the area. The mobileterminal 100 outputs information on a current status of an indicator atthe first area 111. For example, when it is assumed that the first area111 includes an indicator about remaining capacity of a battery and atouch event is generated at the first area 111, the mobile terminal 100outputs a voice signal such as “This is an indicator area and currentremaining capacity of battery is 80%.” After that, when the user inputsa voice signal corresponding to “Check,” the mobile terminal 100supplies electric power to the display unit 141 such that the user maycheck the indicator area.

In a state as shown in the screen 331, when the user generates a presettouch event at the second area 113, the mobile terminal 100 may outputdescription on a user facility assigned to the second area 113, forexample, a preset voice signal such as “This is a widget area. Currenttime is 10: 30 and widget 1 information, widget 2 information, andwidget 3 information are serviced” and information serviced by the userfacility assigned to the second area 113 in a voice signal.

In the state of the screen 331, when the user generates a preset touchevent at the third area 115, the mobile terminal 100 outputs a userfacility assigned to the third area 115, for example, a voice signalcorresponding to “This is an unknown message reception alarmingfunction”, and may output a part of information on the unknown messagereception as a voice signal when the unknown message is received. Forexample, when an unknown message of asking after a user is received from“Su-jan” and a preset touch event is generated at the third area 115,the mobile terminal 100 outputs a voice signal such as “This is anunknown message reception alarming function. A message is received fromSu-jan. How are you? . . . ” through the audio processor unit 130. Inthis case, the mobile terminal 100 may not output the voice signal “Thisis an unknown message reception alarming function” when the unknownmessage is received and further outputs a voice signal such as “Unknownmessage is not received” when unknown message is not received. Moreover,when the unknown message is received from “Su-jan”, the mobile terminal100 outputs all or a part of the unknown message in a voice signal atthe state of the screen 331. When the user inputs a voice signalcorresponding to “Check,” the mobile terminal 100 may perform the voicesensing of the voice signal and output a message checking screen forchecking the unknown message based on the voice sensing.

As described above, when a touch event is generated at an area of thetouch panel even when the display unit is not powered, the mobileterminal according to the present invention outputs a voice signalcorresponding to a user facility assigned to the area of the touch paneland performs the voice sensing function for activating the userfacility. Therefore, the mobile terminal restricts a range of sensing acollected voice signal to a range of a user facility corresponding to anarea where the touch event is generated, so that a database range forthe voice sensing can be reduced and precision of the voice sensing canbe improved.

Note that the above-described methods according to the present inventioncan be realized in hardware or as software or computer code that can bestored in a recording medium such as a CD ROM, an RAM, a floppy disk, ahard disk, or a magneto-optical disk or downloaded over a network, sothat the methods described herein can be executed by such software usinga general purpose computer, or a special processor or in programmable ordedicated hardware, such as an ASIC or FPGA. As would be understood inthe art, the computer, the processor or the programmable hardwareinclude memory components, e.g., RAM, ROM, Flash, etc. that may store orreceive software or computer code that when accessed and executed by thecomputer, processor or hardware implement the processing methodsdescribed herein. Although exemplary embodiments of the presentinvention have been described in detail hereinabove, it should beunderstood that many variations and modifications of the basic inventiveconcept herein described, which may appear to those skilled in the art,will still fall within the spirit and scope of the exemplary embodimentsof the present invention as defined in the appended claims.

1. An input processing method of a mobile terminal, comprising:collecting a touch event and a voice signal; at least one of sensing thevoice signal to generate voice sensing data and generating visualinformation text data corresponding to the touch event; correcting atleast one of the voice sensing data by comparing the voice sensing datawith the visual information text data and correcting the visualinformation text data by comparing the visual information text data withthe voice sensing data; and displaying at least one of the correctedvoice sensing data and the corrected visual information text data.
 2. Aninput processing device of a mobile terminal, comprising: a touch panelfor generating a touch event; a microphone for collecting a voicesignal; a control unit for sensing the voice signal to generate voicesensing data, generating visual information text data, comparing thevoice sensing data with the visual information text data to correct thevoice sensing data or comparing the visual information text data withthe voice sensing data to correct the visual information text data; anda display unit for outputting at least one of the corrected voicesensing data and the corrected visual information text data.